Running Heart Rate Prediction Using Deep Learning

In this notebook I attempt to build a deep learning model to predict my own heart rate using values of speed, cadence (steps per minute) and altitude. The data is gathered from 35 runs of differing length and intensity I ran between January and March 2021 and measured with a Polar Vantage M sports watch. The measurements are done once a second and the data is stored in 35 different csv files acquired from Polar Flow.

Data Input and Preprocessing

First, let's load the libraries.

In [48]:
import numpy as np
import csv
import os
import tensorflow.keras as K
from tensorflow.keras import models
from tensorflow.keras import layers
from tensorflow.keras import regularizers
from sklearn.model_selection import train_test_split
from sklearn.preprocessing import StandardScaler
import matplotlib.pyplot as plt
import random
rng = int(random.random() * 10000)
print("Seed was: " + str(rng))
random.seed(rng)
#7364
Seed was: 8350

Next, I define a couple of functions to help with parsing the data from the csv files. The first one gets an input string in the form "hh:mm:ss" and returns the total time in seconds. The second one imputes the missing values of heart rate to be 0. These missing values only really appear at the beginning of some of the runs and thus they can safely be ignored.

In [49]:
def parseTime(timestring):
    if ":" in timestring:
        stringarr = timestring.split(":")
        timeInSeconds = int(stringarr[0])*(60**2) + int(stringarr[1]) * 60 + int(stringarr[2])
        return timeInSeconds
    else:
        return 0
def parseHR(HRstring):
    if len(HRstring) > 0:
        return(int(HRstring))
    else:
        return 0

The next cell loops through the csv files and reads the desired data into a 3-D python list with shape (number of runs, number of features, number of measurements). The csv files contain metadata of the run in the first 3 rows, so they are skipped.

In [50]:
runningdata = []
for filename in os.listdir("Running Data\\S"):
    with open("Running Data\\S\\" + filename) as file:
        reader = csv.reader(file, delimiter = ",")
        rownum = 0
        rdata = []
        for row in reader:
            if rownum >= 3:
                Time = parseTime(row[1])
                HR = parseHR(row[2])
                Speed = float(row[3])
                Cadence = float(row[5])
                Altitude = float(row[6])
                Distance = float(row[8])
                rdata.append([HR, Speed, Cadence, Altitude, Distance, Time])
            rownum += 1
        runningdata.append(rdata)

Let's plot the data from all of the runs using matplotlib. As we can see, the runs contain very different profiles of heart rate: some from long steady runs and some from high intensity interval runs. Thus the model will have a non-trivial task in predicting the heart rate accurately. Of course, many other things than speed, altitude and cadence have an effect on the heart rate, suchs as running conditions and the amount of exercise on previous days.

In [51]:
titledict = {
    0: "HR",
    1: "Speed",
    2: "Cadence",
    3: "Altitude"
}
for runnum, run in enumerate(runningdata):
    fig, axs = plt.subplots(2,2)
    fig.suptitle("Run Number " + str(runnum), fontsize = 20)
    fig.set_size_inches(20, 10)
    plotnum = 0
    for i in range(2):
        for j in range(2):
            axs[i][j].set_title(titledict[plotnum])
            axs[i][j].plot([x[plotnum] for x in run])
            plotnum += 1
    plt.show()

Something strange happened in the end of run #5 (I probably forgot to turn off the recording when I got home) so I will omit the values after 2000 seconds from the data. If there was a lot more data, it probably wouldn't matter that much though. I also impute the zero values of altitude with the previous or if none exists, the next none zero value in the time series.

In [52]:
runningdata[5] = runningdata[5][0:2000]
for run in runningdata:
    prevnonzero = None
    for i, step in enumerate(run):
        imputer = None
        if step[3] == 0:
            if prevnonzero is not None:
                imputer = prevnonzero
            else:
                for j in range(i+1, len(run)):
                    if run[j][3] != 0:
                        imputer = run[j][3]
                        break
            step[3] = imputer
        else:
            prevnonzero = step[3]
               
plt.plot([x[3] for x in runningdata[0]])
plt.suptitle("Imputed altitude for run number 0")
plt.gcf().set_size_inches(20,5)

The runs are sampled randomly into training, validation and testing runs. I used 5 runs for validation, 3 runs for testing and the rest for training. I also define some hyperparameters of the models: the lookback, i.e. the amount of values of speed, altitude and distance the model uses as features (I selected 300 here corresponding to the data from the last five minutes of running) and padding, which controls whether to use padding for the missing values of predictor values for the first five minutes of running in each run.

The data splits are printed below.

In [53]:
validate_runs = random.sample(list(range(len(runningdata))), 5)
test_runs = random.sample([x for x in list(range(len(runningdata))) if x not in validate_runs], 3)

train_runs = [x for x in list(range(len(runningdata))) if x not in validate_runs and x not in test_runs]

lookback_amount = 300
padding = True
print("Train runs: " + str(train_runs))
print("Validate runs: " +str(validate_runs))
print("Test runs: " + str(test_runs))
Train runs: [0, 1, 2, 3, 5, 6, 7, 8, 9, 10, 13, 14, 15, 16, 17, 19, 20, 21, 23, 24, 26, 27, 29, 30, 31, 33, 34]
Validate runs: [12, 32, 4, 11, 25]
Test runs: [18, 22, 28]

Next, I use nested for loops to create the timeseries for training and testing the model. Each timestep in the series contains data from the last 300 steps. As stated above the variable "padding" controls whether to include the first 300 seconds of each run in the dataset using an extreme value (-1e3) padding for the missing values.

In [54]:
targets_train = []
predictors_train = []
targets_test = []
predictors_test = []
run_indices_train = []
run_indices_test = []
targets_validate = []
predictors_validate = []
run_indices_validate = []

for runnum, run in enumerate(runningdata):
    for i, timestep in enumerate(run):
        skip = False
        speeds = []
        cadences = []
        altitudes = []
        if i >= lookback_amount:
            for j in range(i-lookback_amount, i):
                speeds += [run[j][1]]
                cadences += [run[j][2]]
                altitudes += [run[j][3]]
        elif padding:
            for j in range(lookback_amount-i):
                speeds += [-1e3]
                cadences += [-1e3]
                altitudes += [-1e3]
            for j in range(i):
                speeds += [run[j][1]]
                cadences += [run[j][2]]
                altitudes += [run[j][3]]
        else:
            skip = True
        if(not skip):
            current_predictors = speeds + cadences + altitudes
            if(runnum in train_runs):
                    targets_train.append(timestep[0])
                    predictors_train.append(current_predictors)
                    run_indices_train.append(runnum)
            elif(runnum in test_runs):
                targets_test.append(timestep[0])
                predictors_test.append(current_predictors)
                run_indices_test.append(runnum)
            else:
                targets_validate.append(timestep[0])
                predictors_validate.append(current_predictors)
                run_indices_validate.append(runnum)

Here I convert the nested lists to numpy.arrays and reshape them to the keras standard of neural network data.

In [55]:
targets_train = np.array(targets_train)
targets_train = targets_train.reshape((-1, 1))
predictors_train = np.array(predictors_train)
targets_test = np.array(targets_test)
targets_test = targets_test.reshape((-1, 1))
predictors_test = np.array(predictors_test)
targets_validate = np.array(targets_validate)
targets_validate = targets_validate.reshape((-1, 1))
predictors_validate = np.array(predictors_validate)

run_indices_test = np.array(run_indices_test)
run_indices_train = np.array(run_indices_test)
run_indices_validate = np.array(run_indices_validate)


print("Training set")
print(targets_train.shape)
print(predictors_train.shape)
print("Validation set")
print(targets_validate.shape)
print(predictors_validate.shape)
print("Testing set")
print(targets_test.shape)
print(predictors_test.shape)
Training set
(59632, 1)
(59632, 900)
Validation set
(12865, 1)
(12865, 900)
Testing set
(8009, 1)
(8009, 900)

I rescale the values of the predictors to have mean 0 and standard deviation 1 to speed up and stabilize training.

In [56]:
scaler = StandardScaler()
predictors_train = scaler.fit_transform(predictors_train)
predictors_test =scaler.transform(predictors_test)
predictors_validate =scaler.transform(predictors_validate)

predictors_validate = predictors_validate.reshape(predictors_validate.shape[0], -1, 1)
predictors_test = predictors_test.reshape(predictors_test.shape[0], -1, 1)
predictors_train = predictors_train.reshape(predictors_train.shape[0], -1, 1)

Modeling

The modeling framework is as follows:

  1. Train models using the training set
  2. Choose the best model by validation set performance
  3. Test the chosen model on the test set

The metric which the best model is chosen with is mse which is also used as the loss function.

Zero Hidden Layer Baseline

First, I define the base level of accuracy for the deep learning model by constructing a zero hidden layer linear model. The linear model gives a reasonable baseline of performance for the more complex model to improve upon.

In [57]:
baseModel = models.Sequential()
baseModel.add(layers.Flatten())
baseModel.add(layers.Dense(1))
baseModel.build(predictors_train.shape)
early_stop = K.callbacks.EarlyStopping(
    monitor="val_loss",
    min_delta=0.1,
    patience=15,
    verbose=0,
    mode="min",
    restore_best_weights=True,
)
baseModel.compile(optimizer = K.optimizers.Adam(learning_rate = 0.01), loss = K.losses.MeanSquaredError(), metrics = ["mae"])

The linear model simply minimizes the cost function $\frac{1}{n} \sum_{j=1}^n (y-(\sum_{i=1}^{m}w_ix_i + b))^20)$, where n is the number of samples, m is the number of features and y is the value of the target variable The linear model has 901 parameters corresponding to $300\times 3$ weights of the predictors and 1 bias term. The model is trained using batches of 128 for 150 epochs. The loss function used is mean squared error and the Adaptive Moments Estimation algorithm with learning rate 0.01 is used for optimization. The mean absolute error is also monitored for curiosity's sake. I apply early stopping to for validation mse to find the best model with patience of 15 epochs.

In [58]:
baseModel.summary()
Model: "sequential_5"
_________________________________________________________________
Layer (type)                 Output Shape              Param #   
=================================================================
flatten_5 (Flatten)          multiple                  0         
_________________________________________________________________
dense_10 (Dense)             multiple                  901       
=================================================================
Total params: 901
Trainable params: 901
Non-trainable params: 0
_________________________________________________________________
In [59]:
baseFit = baseModel.fit(predictors_train, targets_train, validation_data = (predictors_validate, targets_validate), batch_size = 128, epochs = 150, shuffle = True, callbacks = [early_stop])
Train on 59632 samples, validate on 12865 samples
Epoch 1/150
59632/59632 [==============================] - 2s 28us/sample - loss: 17652.6658 - mae: 130.8293 - val_loss: 15803.0248 - val_mae: 123.9394
Epoch 2/150
59632/59632 [==============================] - 1s 20us/sample - loss: 16403.7498 - mae: 126.3060 - val_loss: 14748.4603 - val_mae: 119.5896
Epoch 3/150
59632/59632 [==============================] - 1s 19us/sample - loss: 15241.2208 - mae: 121.7919 - val_loss: 13559.7833 - val_mae: 114.4646
Epoch 4/150
59632/59632 [==============================] - 1s 19us/sample - loss: 14151.4628 - mae: 117.3363 - val_loss: 12562.2688 - val_mae: 110.4128
Epoch 5/150
59632/59632 [==============================] - 1s 19us/sample - loss: 13125.8957 - mae: 112.9880 - val_loss: 11580.6251 - val_mae: 105.9527
Epoch 6/150
59632/59632 [==============================] - 1s 20us/sample - loss: 12145.5178 - mae: 108.6735 - val_loss: 10685.0902 - val_mae: 101.6250
Epoch 7/150
59632/59632 [==============================] - 1s 19us/sample - loss: 11214.9357 - mae: 104.3512 - val_loss: 9797.4118 - val_mae: 97.1397
Epoch 8/150
59632/59632 [==============================] - 1s 19us/sample - loss: 10329.2224 - mae: 100.1019 - val_loss: 8967.4009 - val_mae: 92.9306
Epoch 9/150
59632/59632 [==============================] - 1s 19us/sample - loss: 9486.5484 - mae: 95.8456 - val_loss: 8189.5720 - val_mae: 88.5230
Epoch 10/150
59632/59632 [==============================] - 1s 20us/sample - loss: 8689.4884 - mae: 91.5959 - val_loss: 7471.9541 - val_mae: 84.6982
Epoch 11/150
59632/59632 [==============================] - 1s 19us/sample - loss: 7924.9898 - mae: 87.4319 - val_loss: 6749.8333 - val_mae: 80.1744
Epoch 12/150
59632/59632 [==============================] - 1s 20us/sample - loss: 7209.9376 - mae: 83.3104 - val_loss: 6193.8327 - val_mae: 76.5950
Epoch 13/150
59632/59632 [==============================] - 1s 19us/sample - loss: 6534.5272 - mae: 79.1969 - val_loss: 5498.6409 - val_mae: 72.2028
Epoch 14/150
59632/59632 [==============================] - 1s 19us/sample - loss: 5891.8619 - mae: 75.0392 - val_loss: 4899.7850 - val_mae: 67.8937
Epoch 15/150
59632/59632 [==============================] - 1s 19us/sample - loss: 5295.4248 - mae: 70.9630 - val_loss: 4417.6032 - val_mae: 64.1549
Epoch 16/150
59632/59632 [==============================] - 1s 19us/sample - loss: 4729.7643 - mae: 66.8737 - val_loss: 3842.2435 - val_mae: 59.5981
Epoch 17/150
59632/59632 [==============================] - 1s 19us/sample - loss: 4207.8262 - mae: 62.8845 - val_loss: 3400.7306 - val_mae: 55.9106
Epoch 18/150
59632/59632 [==============================] - 1s 19us/sample - loss: 3719.0648 - mae: 58.9116 - val_loss: 2966.0828 - val_mae: 51.8556
Epoch 19/150
59632/59632 [==============================] - 1s 20us/sample - loss: 3265.0818 - mae: 54.9112 - val_loss: 2537.9251 - val_mae: 47.6184
Epoch 20/150
59632/59632 [==============================] - 1s 20us/sample - loss: 2851.0974 - mae: 51.0009 - val_loss: 2180.0965 - val_mae: 43.6259
Epoch 21/150
59632/59632 [==============================] - 1s 21us/sample - loss: 2470.5438 - mae: 47.1215 - val_loss: 1851.4095 - val_mae: 39.8627
Epoch 22/150
59632/59632 [==============================] - 1s 20us/sample - loss: 2128.4642 - mae: 43.3293 - val_loss: 1565.1564 - val_mae: 36.0952
Epoch 23/150
59632/59632 [==============================] - 1s 19us/sample - loss: 1820.0964 - mae: 39.5805 - val_loss: 1314.1381 - val_mae: 32.4118
Epoch 24/150
59632/59632 [==============================] - 1s 19us/sample - loss: 1543.9909 - mae: 35.8845 - val_loss: 1080.0018 - val_mae: 28.6577
Epoch 25/150
59632/59632 [==============================] - 1s 19us/sample - loss: 1300.0846 - mae: 32.3059 - val_loss: 897.3769 - val_mae: 25.0807
Epoch 26/150
59632/59632 [==============================] - 1s 19us/sample - loss: 1089.7344 - mae: 28.9230 - val_loss: 723.9898 - val_mae: 22.0352
Epoch 27/150
59632/59632 [==============================] - 1s 19us/sample - loss: 908.2263 - mae: 25.7383 - val_loss: 596.9387 - val_mae: 19.5034
Epoch 28/150
59632/59632 [==============================] - 1s 19us/sample - loss: 754.6455 - mae: 22.8311 - val_loss: 474.5339 - val_mae: 17.0940
Epoch 29/150
59632/59632 [==============================] - 1s 20us/sample - loss: 630.6200 - mae: 20.2727 - val_loss: 422.6910 - val_mae: 16.2930
Epoch 30/150
59632/59632 [==============================] - 1s 19us/sample - loss: 533.6176 - mae: 18.1547 - val_loss: 346.6240 - val_mae: 14.9893
Epoch 31/150
59632/59632 [==============================] - 1s 19us/sample - loss: 457.9485 - mae: 16.4214 - val_loss: 292.3436 - val_mae: 13.6153
Epoch 32/150
59632/59632 [==============================] - 1s 19us/sample - loss: 405.4381 - mae: 15.1601 - val_loss: 272.9360 - val_mae: 13.5796
Epoch 33/150
59632/59632 [==============================] - 1s 19us/sample - loss: 366.1872 - mae: 14.1844 - val_loss: 257.8370 - val_mae: 13.2358
Epoch 34/150
59632/59632 [==============================] - 1s 19us/sample - loss: 343.6390 - mae: 13.5938 - val_loss: 267.1754 - val_mae: 13.7651
Epoch 35/150
59632/59632 [==============================] - 1s 19us/sample - loss: 331.5214 - mae: 13.2572 - val_loss: 266.9828 - val_mae: 13.8720
Epoch 36/150
59632/59632 [==============================] - 1s 20us/sample - loss: 323.4275 - mae: 13.0474 - val_loss: 267.7162 - val_mae: 13.7352
Epoch 37/150
59632/59632 [==============================] - 1s 20us/sample - loss: 323.3803 - mae: 13.0100 - val_loss: 270.7357 - val_mae: 13.9790
Epoch 38/150
59632/59632 [==============================] - 1s 20us/sample - loss: 321.7580 - mae: 12.9606 - val_loss: 279.8702 - val_mae: 14.1738
Epoch 39/150
59632/59632 [==============================] - 1s 20us/sample - loss: 321.3966 - mae: 12.9463 - val_loss: 304.3971 - val_mae: 14.8535
Epoch 40/150
59632/59632 [==============================] - 1s 19us/sample - loss: 322.2231 - mae: 12.9639 - val_loss: 289.4736 - val_mae: 14.4813
Epoch 41/150
59632/59632 [==============================] - 1s 19us/sample - loss: 322.3091 - mae: 12.9718 - val_loss: 312.6408 - val_mae: 15.1076
Epoch 42/150
59632/59632 [==============================] - ETA: 0s - loss: 321.3793 - mae: 12.94 - 1s 19us/sample - loss: 321.2890 - mae: 12.9432 - val_loss: 277.7125 - val_mae: 14.1802
Epoch 43/150
59632/59632 [==============================] - 1s 20us/sample - loss: 322.0069 - mae: 12.9609 - val_loss: 278.5613 - val_mae: 14.1787
Epoch 44/150
59632/59632 [==============================] - 1s 19us/sample - loss: 321.7498 - mae: 12.9530 - val_loss: 284.4918 - val_mae: 14.3571
Epoch 45/150
59632/59632 [==============================] - 1s 19us/sample - loss: 320.8436 - mae: 12.9182 - val_loss: 282.5185 - val_mae: 14.2689
Epoch 46/150
59632/59632 [==============================] - 1s 19us/sample - loss: 320.2254 - mae: 12.9110 - val_loss: 278.3406 - val_mae: 14.1769
Epoch 47/150
59632/59632 [==============================] - 1s 19us/sample - loss: 322.8135 - mae: 12.9813 - val_loss: 290.4715 - val_mae: 14.5236
Epoch 48/150
59632/59632 [==============================] - 1s 20us/sample - loss: 322.3133 - mae: 12.9594 - val_loss: 295.3847 - val_mae: 14.6703

The training and validation losses as well as the learning curves of the model are printed below. The model achieves the best validation mse (about 257.8) after 33 epochs. The corresponding mean absolute error values is about 13.24. This sets the baseline for model performance.

In [80]:
baseModel.evaluate(predictors_train, targets_train)
baseModel.evaluate(predictors_validate, targets_validate)
fig, axs = plt.subplots(1,2)
fig.set_size_inches(20,5)
axs[0].plot(baseFit.history["loss"], label = "loss")
axs[0].plot(baseFit.history["val_loss"], label = "val loss")
axs[1].plot(baseFit.history["mae"], label = "mae")
axs[1].plot(baseFit.history["val_mae"], label = "val mae")
axs[0].legend()
axs[1].legend()
59632/59632 [==============================] - 2s 38us/sample - loss: 354.3405 - mae: 13.7807
12865/12865 [==============================] - 0s 38us/sample - loss: 257.8370 - mae: 13.2358
Out[80]:
<matplotlib.legend.Legend at 0x21198d3f248>

Here I define a function for printing predictions of the model against the true heart rate values.

In [61]:
def plotPreds(preds, targets, runs, run_indices, labels = ("Model Prediction",)):
    fig, axs = plt.subplots(len(runs))
    fig.set_size_inches(15,15)
    for run, ax in zip(runs, axs):
        ax.plot(targets[run_indices == run, ], label = "Heart Rate")
        for i, pred in enumerate(preds):
            ax.plot(pred[run_indices == run, ], label = labels[i])
        ax.legend()
        ax.set_title("Run number " + str(run))
        ax.set_ylim((80, 190))

The predictions of the linear model for the validation set look like this.

In [62]:
basePreds = baseModel.predict(predictors_validate)
plotPreds((basePreds,), targets_validate, validate_runs, run_indices_validate)

The linear model seems to fit the data decently well, although there is some clear room for improvement. Here are the MSE and MAE values for the different validation runs.

In [63]:
for run in validate_runs:
    print("Run number " + str(run))
    baseModel.evaluate(predictors_validate[run_indices_validate == run,], targets_validate[run_indices_validate == run,])
Run number 12
3794/3794 [==============================] - 0s 41us/sample - loss: 203.6485 - mae: 12.7704
Run number 32
2705/2705 [==============================] - 0s 42us/sample - loss: 300.3270 - mae: 14.1018
Run number 4
722/722 [==============================] - 0s 48us/sample - loss: 259.6020 - mae: 13.3098
Run number 11
2635/2635 [==============================] - 0s 45us/sample - loss: 183.1456 - mae: 10.7881
Run number 25
3009/3009 [==============================] - 0s 42us/sample - loss: 352.9493 - mae: 15.1698

Fully Connected Deep Network

Next, I define the model architecture for a deeper neural network. After trying multiple hyperparameters I decided to go for a fully connected network with four hidden layers consisting of 512, 256, 128 and 32 hidden units respectively. The hidden layers use ReLU as their activation function whereas the output layer uses the identity function. I added L1 and L2 regularization to each of the hidden units as well as dropout of 30 % after the hiden layers to prevent overfitting. I chose adaptive moments estimation algorithm with learning rate 0.0005 as the optimizer. The same principle of early stopping is applied as above.

In [64]:
deepModel = models.Sequential()
deepModel.add(layers.Flatten())
deepModel.add(layers.Dense(512, kernel_regularizer = regularizers.l1_l2(l1 = 1e-1, l2 = 1e-1)))
deepModel.add(layers.ReLU())
deepModel.add(layers.Dropout(0.3))
deepModel.add(layers.Dense(256, kernel_regularizer = regularizers.l1_l2(l1 = 1e-1, l2 = 1e-1)))
deepModel.add(layers.ReLU())
deepModel.add(layers.Dropout(0.3))
deepModel.add(layers.Dense(128, kernel_regularizer = regularizers.l1_l2(l1 = 1e-1, l2 = 1e-1)))
deepModel.add(layers.ReLU())
deepModel.add(layers.Dropout(0.3))
deepModel.add(layers.Dense(32, kernel_regularizer = regularizers.l1_l2(l1 = 1e-1, l2 = 1e-1)))
deepModel.add(layers.ReLU())
deepModel.add(layers.Dense(1))


deepModel.build(predictors_train.shape)
deepModel.compile(
    optimizer=K.optimizers.Adam(learning_rate=5e-4), loss = K.losses.MeanSquaredError(), metrics = ["mae"])

The model has 629 697 trainable parameters, divided into layers as listed below.

In [65]:
deepModel.summary()
Model: "sequential_6"
_________________________________________________________________
Layer (type)                 Output Shape              Param #   
=================================================================
flatten_6 (Flatten)          multiple                  0         
_________________________________________________________________
dense_11 (Dense)             multiple                  461312    
_________________________________________________________________
re_lu_7 (ReLU)               multiple                  0         
_________________________________________________________________
dropout_6 (Dropout)          multiple                  0         
_________________________________________________________________
dense_12 (Dense)             multiple                  131328    
_________________________________________________________________
re_lu_8 (ReLU)               multiple                  0         
_________________________________________________________________
dropout_7 (Dropout)          multiple                  0         
_________________________________________________________________
dense_13 (Dense)             multiple                  32896     
_________________________________________________________________
re_lu_9 (ReLU)               multiple                  0         
_________________________________________________________________
dropout_8 (Dropout)          multiple                  0         
_________________________________________________________________
dense_14 (Dense)             multiple                  4128      
_________________________________________________________________
re_lu_10 (ReLU)              multiple                  0         
_________________________________________________________________
dense_15 (Dense)             multiple                  33        
=================================================================
Total params: 629,697
Trainable params: 629,697
Non-trainable params: 0
_________________________________________________________________

I train the model using a batch size of 128 with a maximum number of epochs at 500.

In [66]:
deepFit = deepModel.fit(predictors_train, targets_train, validation_data = (predictors_validate, targets_validate), batch_size = 128, epochs = 500, shuffle = True, callbacks = [early_stop])
Train on 59632 samples, validate on 12865 samples
Epoch 1/500
59632/59632 [==============================] - 2s 40us/sample - loss: 2855.2088 - mae: 21.9171 - val_loss: 1313.5833 - val_mae: 11.4340
Epoch 2/500
59632/59632 [==============================] - 2s 27us/sample - loss: 1244.6013 - mae: 15.6357 - val_loss: 772.5588 - val_mae: 11.2685
Epoch 3/500
59632/59632 [==============================] - 2s 27us/sample - loss: 911.5334 - mae: 15.3747 - val_loss: 597.8884 - val_mae: 11.4051
Epoch 4/500
59632/59632 [==============================] - 2s 26us/sample - loss: 776.3053 - mae: 15.2306 - val_loss: 553.9004 - val_mae: 12.4522
Epoch 5/500
59632/59632 [==============================] - 2s 26us/sample - loss: 702.0504 - mae: 15.0421 - val_loss: 436.1777 - val_mae: 10.6749
Epoch 6/500
59632/59632 [==============================] - 2s 25us/sample - loss: 655.4132 - mae: 14.9166 - val_loss: 396.8742 - val_mae: 10.5830
Epoch 7/500
59632/59632 [==============================] - 2s 26us/sample - loss: 621.2851 - mae: 14.8270 - val_loss: 399.9351 - val_mae: 11.2571
Epoch 8/500
59632/59632 [==============================] - 2s 26us/sample - loss: 594.7330 - mae: 14.7251 - val_loss: 448.2426 - val_mae: 12.6451
Epoch 9/500
59632/59632 [==============================] - 2s 26us/sample - loss: 562.4764 - mae: 14.4198 - val_loss: 384.9527 - val_mae: 12.1352
Epoch 10/500
59632/59632 [==============================] - 2s 27us/sample - loss: 543.8272 - mae: 14.2798 - val_loss: 354.4852 - val_mae: 11.4879
Epoch 11/500
59632/59632 [==============================] - 2s 27us/sample - loss: 513.6730 - mae: 13.9440 - val_loss: 371.8507 - val_mae: 12.4085
Epoch 12/500
59632/59632 [==============================] - 2s 26us/sample - loss: 489.4709 - mae: 13.6614 - val_loss: 371.1204 - val_mae: 12.7105
Epoch 13/500
59632/59632 [==============================] - 2s 26us/sample - loss: 471.9522 - mae: 13.4628 - val_loss: 341.4123 - val_mae: 12.0860
Epoch 14/500
59632/59632 [==============================] - 2s 26us/sample - loss: 451.9432 - mae: 13.2300 - val_loss: 387.6726 - val_mae: 13.5722
Epoch 15/500
59632/59632 [==============================] - 2s 26us/sample - loss: 435.9854 - mae: 13.1168 - val_loss: 384.6519 - val_mae: 13.6483
Epoch 16/500
59632/59632 [==============================] - 2s 27us/sample - loss: 420.3680 - mae: 12.9808 - val_loss: 513.8243 - val_mae: 16.8189
Epoch 17/500
59632/59632 [==============================] - 2s 26us/sample - loss: 417.9521 - mae: 12.9895 - val_loss: 414.1248 - val_mae: 14.5482
Epoch 18/500
59632/59632 [==============================] - 2s 26us/sample - loss: 414.1809 - mae: 12.9674 - val_loss: 398.3486 - val_mae: 14.2464
Epoch 19/500
59632/59632 [==============================] - 2s 27us/sample - loss: 405.8477 - mae: 12.8367 - val_loss: 451.6865 - val_mae: 15.5410
Epoch 20/500
59632/59632 [==============================] - 2s 27us/sample - loss: 398.3215 - mae: 12.7968 - val_loss: 372.7308 - val_mae: 13.8464
Epoch 21/500
59632/59632 [==============================] - 2s 27us/sample - loss: 392.2542 - mae: 12.6626 - val_loss: 457.9552 - val_mae: 15.8334
Epoch 22/500
59632/59632 [==============================] - 2s 27us/sample - loss: 389.2384 - mae: 12.6322 - val_loss: 371.5626 - val_mae: 13.8696
Epoch 23/500
59632/59632 [==============================] - 2s 27us/sample - loss: 390.0042 - mae: 12.6204 - val_loss: 378.9421 - val_mae: 13.9175
Epoch 24/500
59632/59632 [==============================] - 2s 27us/sample - loss: 385.3836 - mae: 12.5835 - val_loss: 428.1013 - val_mae: 15.2754
Epoch 25/500
59632/59632 [==============================] - 2s 27us/sample - loss: 386.0726 - mae: 12.6024 - val_loss: 275.9505 - val_mae: 11.6904
Epoch 26/500
59632/59632 [==============================] - 2s 26us/sample - loss: 376.9993 - mae: 12.4863 - val_loss: 415.2156 - val_mae: 14.9646
Epoch 27/500
59632/59632 [==============================] - 2s 26us/sample - loss: 381.0591 - mae: 12.5077 - val_loss: 311.6737 - val_mae: 12.5611
Epoch 28/500
59632/59632 [==============================] - 2s 27us/sample - loss: 372.6570 - mae: 12.3512 - val_loss: 321.1700 - val_mae: 12.7963
Epoch 29/500
59632/59632 [==============================] - 2s 26us/sample - loss: 371.7712 - mae: 12.3713 - val_loss: 342.8510 - val_mae: 13.3931
Epoch 30/500
59632/59632 [==============================] - 2s 27us/sample - loss: 369.1773 - mae: 12.3779 - val_loss: 280.5051 - val_mae: 11.8837
Epoch 31/500
59632/59632 [==============================] - 2s 26us/sample - loss: 366.7812 - mae: 12.2879 - val_loss: 315.6198 - val_mae: 12.7717
Epoch 32/500
59632/59632 [==============================] - 2s 26us/sample - loss: 366.7727 - mae: 12.3295 - val_loss: 363.6188 - val_mae: 13.9021
Epoch 33/500
59632/59632 [==============================] - 2s 27us/sample - loss: 363.8930 - mae: 12.2702 - val_loss: 284.0767 - val_mae: 11.8429
Epoch 34/500
59632/59632 [==============================] - 2s 26us/sample - loss: 364.3728 - mae: 12.2363 - val_loss: 347.6071 - val_mae: 13.5102
Epoch 35/500
59632/59632 [==============================] - 2s 26us/sample - loss: 357.8868 - mae: 12.1299 - val_loss: 273.1597 - val_mae: 11.7803
Epoch 36/500
59632/59632 [==============================] - 2s 28us/sample - loss: 365.5327 - mae: 12.2308 - val_loss: 241.7661 - val_mae: 10.8401
Epoch 37/500
59632/59632 [==============================] - 2s 28us/sample - loss: 362.5433 - mae: 12.2346 - val_loss: 250.0891 - val_mae: 11.1143
Epoch 38/500
59632/59632 [==============================] - 2s 28us/sample - loss: 358.1123 - mae: 12.1518 - val_loss: 282.0806 - val_mae: 12.0341
Epoch 39/500
59632/59632 [==============================] - 2s 27us/sample - loss: 355.9105 - mae: 12.1260 - val_loss: 259.4609 - val_mae: 11.3965
Epoch 40/500
59632/59632 [==============================] - 2s 27us/sample - loss: 354.2197 - mae: 12.1342 - val_loss: 290.1686 - val_mae: 12.1384
Epoch 41/500
59632/59632 [==============================] - 2s 26us/sample - loss: 352.4498 - mae: 12.0798 - val_loss: 307.1411 - val_mae: 12.6750
Epoch 42/500
59632/59632 [==============================] - 2s 26us/sample - loss: 353.0953 - mae: 12.0609 - val_loss: 223.7896 - val_mae: 10.4490
Epoch 43/500
59632/59632 [==============================] - 2s 26us/sample - loss: 351.7032 - mae: 12.0762 - val_loss: 238.6848 - val_mae: 10.9366
Epoch 44/500
59632/59632 [==============================] - 2s 26us/sample - loss: 351.8276 - mae: 12.0664 - val_loss: 336.7576 - val_mae: 13.2305
Epoch 45/500
59632/59632 [==============================] - 2s 26us/sample - loss: 348.1694 - mae: 12.0422 - val_loss: 290.2027 - val_mae: 12.2625
Epoch 46/500
59632/59632 [==============================] - 2s 26us/sample - loss: 347.7073 - mae: 12.0055 - val_loss: 286.3427 - val_mae: 12.1288
Epoch 47/500
59632/59632 [==============================] - 2s 26us/sample - loss: 347.6153 - mae: 12.0090 - val_loss: 287.9169 - val_mae: 12.1909
Epoch 48/500
59632/59632 [==============================] - 2s 26us/sample - loss: 348.1491 - mae: 12.0088 - val_loss: 237.1205 - val_mae: 10.8718
Epoch 49/500
59632/59632 [==============================] - 2s 26us/sample - loss: 348.1242 - mae: 12.0088 - val_loss: 299.6357 - val_mae: 12.3975
Epoch 50/500
59632/59632 [==============================] - 2s 25us/sample - loss: 344.2193 - mae: 11.9431 - val_loss: 270.5924 - val_mae: 11.8047
Epoch 51/500
59632/59632 [==============================] - 2s 26us/sample - loss: 343.5021 - mae: 11.8976 - val_loss: 260.2909 - val_mae: 11.5579
Epoch 52/500
59632/59632 [==============================] - 2s 26us/sample - loss: 345.8554 - mae: 11.9522 - val_loss: 238.4972 - val_mae: 10.9322
Epoch 53/500
59632/59632 [==============================] - 2s 26us/sample - loss: 340.9896 - mae: 11.8842 - val_loss: 216.6137 - val_mae: 10.3372
Epoch 54/500
59632/59632 [==============================] - 2s 26us/sample - loss: 341.9173 - mae: 11.8888 - val_loss: 248.1886 - val_mae: 11.3106
Epoch 55/500
59632/59632 [==============================] - 2s 26us/sample - loss: 339.3154 - mae: 11.8241 - val_loss: 234.0552 - val_mae: 10.8622
Epoch 56/500
59632/59632 [==============================] - 2s 26us/sample - loss: 339.5944 - mae: 11.8618 - val_loss: 237.3911 - val_mae: 10.9889
Epoch 57/500
59632/59632 [==============================] - 2s 26us/sample - loss: 340.1902 - mae: 11.8705 - val_loss: 256.3219 - val_mae: 11.4493
Epoch 58/500
59632/59632 [==============================] - 2s 26us/sample - loss: 335.1248 - mae: 11.7461 - val_loss: 235.3338 - val_mae: 11.0035
Epoch 59/500
59632/59632 [==============================] - 2s 26us/sample - loss: 338.3567 - mae: 11.8569 - val_loss: 243.4808 - val_mae: 11.1591
Epoch 60/500
59632/59632 [==============================] - 2s 26us/sample - loss: 336.5858 - mae: 11.8186 - val_loss: 218.1621 - val_mae: 10.4698
Epoch 61/500
59632/59632 [==============================] - 2s 26us/sample - loss: 336.7454 - mae: 11.8105 - val_loss: 258.7747 - val_mae: 11.5773
Epoch 62/500
59632/59632 [==============================] - 2s 26us/sample - loss: 334.8775 - mae: 11.7435 - val_loss: 250.5038 - val_mae: 11.3238
Epoch 63/500
59632/59632 [==============================] - 2s 28us/sample - loss: 334.5766 - mae: 11.7386 - val_loss: 232.4548 - val_mae: 10.8371
Epoch 64/500
59632/59632 [==============================] - 2s 27us/sample - loss: 333.6898 - mae: 11.7419 - val_loss: 257.4884 - val_mae: 11.5635
Epoch 65/500
59632/59632 [==============================] - 2s 27us/sample - loss: 333.8415 - mae: 11.7492 - val_loss: 268.3739 - val_mae: 11.7768
Epoch 66/500
59632/59632 [==============================] - 2s 28us/sample - loss: 334.5715 - mae: 11.7954 - val_loss: 271.5078 - val_mae: 11.9559
Epoch 67/500
59632/59632 [==============================] - 2s 27us/sample - loss: 334.0177 - mae: 11.7718 - val_loss: 220.6743 - val_mae: 10.5512
Epoch 68/500
59632/59632 [==============================] - 2s 27us/sample - loss: 333.4534 - mae: 11.7171 - val_loss: 227.9340 - val_mae: 10.8483

More or less surprisingly the model achieves a better MSE value for the validation data than the training data. This would imply that the validation data had perhaps "easier" values of heart rate to predict but the same phenomenon could be observed for different train/test splits as well. The model achieved the best validation MSE of about 216.6 after 63 epochs, a solid increase on the base model. The corresponding MAE value is about 10.34. Here are the loss values plotted for the training and testing sets per epoch:

In [81]:
deepModel.evaluate(predictors_train, targets_train)
deepModel.evaluate(predictors_validate, targets_validate)
fig, axs = plt.subplots(1,2)
fig.set_size_inches(20, 5)
axs[0].plot(deepFit.history["loss"], label = "loss")
axs[0].plot(deepFit.history["val_loss"], label = "val loss")
axs[1].plot(deepFit.history["mae"], label = "mae")
axs[1].plot(deepFit.history["val_mae"], label = "val mae")
axs[0].legend()
axs[1].legend()
59632/59632 [==============================] - 3s 50us/sample - loss: 321.0060 - mae: 11.7228
12865/12865 [==============================] - 1s 48us/sample - loss: 216.6137 - mae: 10.3372
Out[81]:
<matplotlib.legend.Legend at 0x20fc007c248>

Here are the predictions for the runs in the validation dataset plotted with the real heart rate values. The plot seem to show that the model is better in predicting the changes in heart-rate

In [68]:
predsDeep = deepModel.predict(predictors_validate)
plotPreds((predsDeep,), targets_validate, validate_runs, run_indices_validate)

Here, the model performance is evaluated seperately for the validation runs. Run number 4 seems to stand out having a lot higher MSE value than the other runs.

In [82]:
for run in validate_runs:
    print("Run number " + str(run))
    deepModel.evaluate(predictors_validate[run_indices_validate == run,], targets_validate[run_indices_validate == run,])
Run number 12
3794/3794 [==============================] - 0s 55us/sample - loss: 180.0725 - mae: 9.3114
Run number 32
2705/2705 [==============================] - 0s 50us/sample - loss: 277.3524 - mae: 12.6333
Run number 4
722/722 [==============================] - 0s 62us/sample - loss: 403.4653 - mae: 16.4632
Run number 11
2635/2635 [==============================] - 0s 50us/sample - loss: 181.0911 - mae: 8.4003
Run number 25
3009/3009 [==============================] - 0s 51us/sample - loss: 194.3587 - mae: 9.7926

Convolutional Model

Finally, I define a convolutional neural network architecture. Using a convolutional model makes sense because the convolutional layers can use features learnt from different parts of the input in other places. An increase in the altitude values (an uphill) should have an increasing effect to the heart rate values regardless of whether it happened 20 or 100 seconds ago. The convolutional network has two convolutional layers followed by one hidden fully connected layer, totaling to about 2.7 million parameters. The complete model description is listed below. The Adam algorithm with learning rate 0.0001 is used with a batch size of 128. The same process is also used for early stopping.

In [70]:
convModel = models.Sequential()
convModel.add(layers.Conv1D(50, 60, 1))
convModel.add(layers.ReLU())
convModel.add(layers.Dropout(0.2))
convModel.add(layers.Conv1D(100, 20, 1))
convModel.add(layers.ReLU())
convModel.add(layers.Dropout(0.2))
convModel.add(layers.Flatten())
convModel.add(layers.Dense(32))
convModel.add(layers.ReLU())
convModel.add(layers.Dropout(0.3))
convModel.add(layers.Dense(1))



convModel.build(predictors_train.shape)
convModel.compile(
    optimizer=K.optimizers.Adam(learning_rate=1e-4),
    loss = K.losses.MeanSquaredError(), metrics = ["mae"])
In [71]:
convModel.summary()
Model: "sequential_7"
_________________________________________________________________
Layer (type)                 Output Shape              Param #   
=================================================================
conv1d_2 (Conv1D)            multiple                  3050      
_________________________________________________________________
re_lu_11 (ReLU)              multiple                  0         
_________________________________________________________________
dropout_9 (Dropout)          multiple                  0         
_________________________________________________________________
conv1d_3 (Conv1D)            multiple                  100100    
_________________________________________________________________
re_lu_12 (ReLU)              multiple                  0         
_________________________________________________________________
dropout_10 (Dropout)         multiple                  0         
_________________________________________________________________
flatten_7 (Flatten)          multiple                  0         
_________________________________________________________________
dense_16 (Dense)             multiple                  2630432   
_________________________________________________________________
re_lu_13 (ReLU)              multiple                  0         
_________________________________________________________________
dropout_11 (Dropout)         multiple                  0         
_________________________________________________________________
dense_17 (Dense)             multiple                  33        
=================================================================
Total params: 2,733,615
Trainable params: 2,733,615
Non-trainable params: 0
_________________________________________________________________
In [72]:
convFit = convModel.fit(predictors_train, targets_train, validation_data = (predictors_validate, targets_validate), batch_size = 128, epochs = 500, shuffle = True, callbacks = [early_stop])
Train on 59632 samples, validate on 12865 samples
Epoch 1/500
59632/59632 [==============================] - 9s 159us/sample - loss: 1862.5481 - mae: 31.0351 - val_loss: 165.0910 - val_mae: 10.6300
Epoch 2/500
59632/59632 [==============================] - 9s 145us/sample - loss: 953.4777 - mae: 23.8659 - val_loss: 183.6719 - val_mae: 11.0452
Epoch 3/500
59632/59632 [==============================] - 9s 145us/sample - loss: 934.3423 - mae: 23.6555 - val_loss: 154.3185 - val_mae: 10.2604
Epoch 4/500
59632/59632 [==============================] - 9s 145us/sample - loss: 934.9977 - mae: 23.6104 - val_loss: 159.5680 - val_mae: 10.4382
Epoch 5/500
59632/59632 [==============================] - 9s 146us/sample - loss: 925.7552 - mae: 23.4786 - val_loss: 150.6191 - val_mae: 10.0951
Epoch 6/500
59632/59632 [==============================] - 9s 145us/sample - loss: 926.0215 - mae: 23.4239 - val_loss: 148.1943 - val_mae: 9.9852
Epoch 7/500
59632/59632 [==============================] - 9s 145us/sample - loss: 916.0640 - mae: 23.3522 - val_loss: 139.7751 - val_mae: 9.7467
Epoch 8/500
59632/59632 [==============================] - 9s 145us/sample - loss: 915.6870 - mae: 23.3518 - val_loss: 167.0737 - val_mae: 10.5397
Epoch 9/500
59632/59632 [==============================] - 9s 147us/sample - loss: 920.2317 - mae: 23.4151 - val_loss: 152.1037 - val_mae: 10.1323
Epoch 10/500
59632/59632 [==============================] - 9s 145us/sample - loss: 913.9473 - mae: 23.3573 - val_loss: 199.5321 - val_mae: 11.4258
Epoch 11/500
59632/59632 [==============================] - 9s 146us/sample - loss: 925.7772 - mae: 23.4052 - val_loss: 157.4398 - val_mae: 10.2316
Epoch 12/500
59632/59632 [==============================] - 9s 145us/sample - loss: 924.2013 - mae: 23.5010 - val_loss: 169.0354 - val_mae: 10.6446
Epoch 13/500
59632/59632 [==============================] - 9s 145us/sample - loss: 916.4812 - mae: 23.3575 - val_loss: 185.4372 - val_mae: 11.0869
Epoch 14/500
59632/59632 [==============================] - 9s 148us/sample - loss: 925.8966 - mae: 23.5258 - val_loss: 132.1791 - val_mae: 9.4097
Epoch 15/500
59632/59632 [==============================] - 9s 148us/sample - loss: 918.3433 - mae: 23.3689 - val_loss: 134.1540 - val_mae: 9.4687
Epoch 16/500
59632/59632 [==============================] - 9s 147us/sample - loss: 912.5682 - mae: 23.3148 - val_loss: 145.0522 - val_mae: 9.8586
Epoch 17/500
59632/59632 [==============================] - 9s 147us/sample - loss: 917.1749 - mae: 23.3043 - val_loss: 141.8802 - val_mae: 9.7299
Epoch 18/500
59632/59632 [==============================] - 9s 148us/sample - loss: 910.6765 - mae: 23.2616 - val_loss: 141.3586 - val_mae: 9.6616
Epoch 19/500
59632/59632 [==============================] - 9s 148us/sample - loss: 917.6100 - mae: 23.3339 - val_loss: 141.0695 - val_mae: 9.6913
Epoch 20/500
59632/59632 [==============================] - 9s 151us/sample - loss: 905.7741 - mae: 23.2087 - val_loss: 138.7315 - val_mae: 9.6280
Epoch 21/500
59632/59632 [==============================] - 9s 153us/sample - loss: 908.4260 - mae: 23.2651 - val_loss: 131.7974 - val_mae: 9.3181
Epoch 22/500
59632/59632 [==============================] - 10s 159us/sample - loss: 914.8132 - mae: 23.3666 - val_loss: 145.6971 - val_mae: 9.7664
Epoch 23/500
59632/59632 [==============================] - 9s 154us/sample - loss: 905.6373 - mae: 23.1831 - val_loss: 135.9303 - val_mae: 9.4428
Epoch 24/500
59632/59632 [==============================] - 9s 149us/sample - loss: 907.0321 - mae: 23.2085 - val_loss: 129.8877 - val_mae: 9.2679
Epoch 25/500
59632/59632 [==============================] - 9s 144us/sample - loss: 912.0138 - mae: 23.2661 - val_loss: 143.0608 - val_mae: 9.6423
Epoch 26/500
59632/59632 [==============================] - 9s 144us/sample - loss: 903.1320 - mae: 23.1759 - val_loss: 131.1029 - val_mae: 9.2744
Epoch 27/500
59632/59632 [==============================] - 9s 145us/sample - loss: 898.1321 - mae: 23.0773 - val_loss: 129.5262 - val_mae: 9.1788
Epoch 28/500
59632/59632 [==============================] - 9s 144us/sample - loss: 900.6925 - mae: 23.1569 - val_loss: 164.9049 - val_mae: 10.3965
Epoch 29/500
59632/59632 [==============================] - 9s 150us/sample - loss: 903.4255 - mae: 23.1157 - val_loss: 145.8781 - val_mae: 9.7905
Epoch 30/500
59632/59632 [==============================] - 9s 151us/sample - loss: 893.7818 - mae: 23.0619 - val_loss: 141.3601 - val_mae: 9.5613
Epoch 31/500
59632/59632 [==============================] - 9s 149us/sample - loss: 892.3090 - mae: 23.0303 - val_loss: 132.2408 - val_mae: 9.2167
Epoch 32/500
59632/59632 [==============================] - 9s 151us/sample - loss: 896.6195 - mae: 23.0929 - val_loss: 124.3626 - val_mae: 8.9523
Epoch 33/500
59632/59632 [==============================] - 9s 149us/sample - loss: 896.8052 - mae: 23.1075 - val_loss: 130.9501 - val_mae: 9.1939
Epoch 34/500
59632/59632 [==============================] - 9s 149us/sample - loss: 893.5612 - mae: 23.0267 - val_loss: 135.2597 - val_mae: 9.2955
Epoch 35/500
59632/59632 [==============================] - 9s 149us/sample - loss: 898.3948 - mae: 23.0909 - val_loss: 134.5328 - val_mae: 9.3164
Epoch 36/500
59632/59632 [==============================] - 9s 150us/sample - loss: 898.7235 - mae: 23.0817 - val_loss: 140.3581 - val_mae: 9.4964
Epoch 37/500
59632/59632 [==============================] - 9s 149us/sample - loss: 892.6455 - mae: 22.9939 - val_loss: 126.0116 - val_mae: 8.9420
Epoch 38/500
59632/59632 [==============================] - 9s 149us/sample - loss: 882.4269 - mae: 22.8923 - val_loss: 195.9482 - val_mae: 11.2446
Epoch 39/500
59632/59632 [==============================] - 9s 149us/sample - loss: 890.2786 - mae: 22.9417 - val_loss: 126.3188 - val_mae: 8.9531
Epoch 40/500
59632/59632 [==============================] - 9s 150us/sample - loss: 888.3334 - mae: 22.9883 - val_loss: 186.4980 - val_mae: 11.0768
Epoch 41/500
59632/59632 [==============================] - 9s 151us/sample - loss: 894.6949 - mae: 23.0610 - val_loss: 133.5065 - val_mae: 9.2245
Epoch 42/500
59632/59632 [==============================] - 9s 148us/sample - loss: 890.9691 - mae: 22.9766 - val_loss: 135.2265 - val_mae: 9.2685
Epoch 43/500
59632/59632 [==============================] - 9s 148us/sample - loss: 887.0194 - mae: 22.9617 - val_loss: 136.8911 - val_mae: 9.3492
Epoch 44/500
59632/59632 [==============================] - 9s 149us/sample - loss: 880.9182 - mae: 22.9036 - val_loss: 124.4374 - val_mae: 8.8719
Epoch 45/500
59632/59632 [==============================] - 9s 144us/sample - loss: 887.2119 - mae: 22.8856 - val_loss: 169.0574 - val_mae: 10.4278
Epoch 46/500
59632/59632 [==============================] - 9s 144us/sample - loss: 880.8253 - mae: 22.9282 - val_loss: 121.9345 - val_mae: 8.6938
Epoch 47/500
59632/59632 [==============================] - 9s 144us/sample - loss: 890.3750 - mae: 22.9653 - val_loss: 121.2033 - val_mae: 8.6915
Epoch 48/500
59632/59632 [==============================] - 9s 143us/sample - loss: 886.2730 - mae: 22.9126 - val_loss: 156.9943 - val_mae: 10.0646
Epoch 49/500
59632/59632 [==============================] - 9s 144us/sample - loss: 875.4542 - mae: 22.7830 - val_loss: 134.8587 - val_mae: 9.2349
Epoch 50/500
59632/59632 [==============================] - 9s 143us/sample - loss: 878.9943 - mae: 22.8703 - val_loss: 125.5989 - val_mae: 8.8269
Epoch 51/500
59632/59632 [==============================] - 9s 144us/sample - loss: 877.4313 - mae: 22.8126 - val_loss: 130.3685 - val_mae: 9.0450
Epoch 52/500
59632/59632 [==============================] - 9s 144us/sample - loss: 869.1244 - mae: 22.7477 - val_loss: 175.7188 - val_mae: 10.6575
Epoch 53/500
59632/59632 [==============================] - 9s 143us/sample - loss: 884.3236 - mae: 22.9223 - val_loss: 173.5764 - val_mae: 10.6661
Epoch 54/500
59632/59632 [==============================] - 9s 143us/sample - loss: 875.8242 - mae: 22.7373 - val_loss: 130.2483 - val_mae: 9.0355
Epoch 55/500
59632/59632 [==============================] - 9s 147us/sample - loss: 875.1378 - mae: 22.8105 - val_loss: 136.4478 - val_mae: 9.2952
Epoch 56/500
59632/59632 [==============================] - 9s 150us/sample - loss: 879.0528 - mae: 22.8664 - val_loss: 127.9733 - val_mae: 8.9225
Epoch 57/500
59632/59632 [==============================] - 9s 150us/sample - loss: 875.7018 - mae: 22.8026 - val_loss: 123.4275 - val_mae: 8.7905
Epoch 58/500
59632/59632 [==============================] - 9s 147us/sample - loss: 868.4820 - mae: 22.6940 - val_loss: 175.6470 - val_mae: 10.7108
Epoch 59/500
59632/59632 [==============================] - 9s 148us/sample - loss: 877.4488 - mae: 22.8383 - val_loss: 149.6170 - val_mae: 9.7474
Epoch 60/500
59632/59632 [==============================] - 9s 144us/sample - loss: 871.5598 - mae: 22.7520 - val_loss: 126.5351 - val_mae: 8.7622
Epoch 61/500
59632/59632 [==============================] - 9s 148us/sample - loss: 874.2946 - mae: 22.7806 - val_loss: 214.2528 - val_mae: 11.8308
Epoch 62/500
59632/59632 [==============================] - 9s 149us/sample - loss: 872.1235 - mae: 22.8001 - val_loss: 137.6589 - val_mae: 9.2381

Again, the validation set gets a lower MSE, clocking in at 121.20 after 57 epochs. The corresponding MAE value is 8.69, a significant increase.

In [83]:
convModel.evaluate(predictors_train, targets_train)
convModel.evaluate(predictors_validate, targets_validate)
fig, axs = plt.subplots(1,2)
fig.set_size_inches(20,5)
axs[0].plot(convFit.history["loss"], label = "loss")
axs[0].plot(convFit.history["val_loss"], label = "val loss")
axs[1].plot(convFit.history["mae"], label = "mae")
axs[1].plot(convFit.history["val_mae"], label = "val mae")
axs[0].legend()
axs[1].legend()
59632/59632 [==============================] - 5s 86us/sample - loss: 217.4109 - mae: 9.9354
12865/12865 [==============================] - 1s 88us/sample - loss: 121.2033 - mae: 8.6915
Out[83]:
<matplotlib.legend.Legend at 0x20fbeb25c48>

Here are the predictions plotted for the validation runs. All of the runs have an improved MSE value compared to the fully connected network. Run #4 though seems to stand out again with a high MSE value. Based on these observations the convolutional model is chosen as the best model.

In [74]:
predsConv = convModel.predict(predictors_validate)
plotPreds((predsConv,), targets_validate, validate_runs, run_indices_validate)
In [84]:
for run in validate_runs:
    print("Run number " + str(run))
    convModel.evaluate(predictors_validate[run_indices_validate == run,], targets_validate[run_indices_validate == run,])
Run number 12
3794/3794 [==============================] - 0s 92us/sample - loss: 101.8348 - mae: 8.1597
Run number 32
2705/2705 [==============================] - 0s 89us/sample - loss: 153.0710 - mae: 10.3283
Run number 4
722/722 [==============================] - 0s 99us/sample - loss: 320.8531 - mae: 14.5676
Run number 11
2635/2635 [==============================] - 0s 93us/sample - loss: 117.0254 - mae: 8.2386
Run number 25
3009/3009 [==============================] - 0s 93us/sample - loss: 72.7298 - mae: 6.8773

Here are the predictions for the different models plotted against the true heart rate values in the validation runs.

In [94]:
plotPreds((basePreds, predsDeep, predsConv), targets_validate, validate_runs, run_indices_validate, ("Base", "Deep FC", "CONV"))

Testing

Here, I test the chosen best model, the convolutional model on the testing set runs. The model achieves an MSE of about 143.28, a minor decrease on the validation MSE. The corresponding MAE value is about 9.61.

In [85]:
convModel.evaluate(predictors_test, targets_test)
8009/8009 [==============================] - 1s 90us/sample - loss: 143.2774 - mae: 9.6055
Out[85]:
[143.27741208605173, 9.605536]

The prediction plots for the test set are below. Apart from the mid portion of run #18, the predictions seem quite accurate.

In [88]:
preds_test = convModel.predict(predictors_test)
plotPreds((preds_test,), targets_test, test_runs, run_indices_test)
In [95]:
for run in test_runs:
    print("Run number " + str(run))
    convModel.evaluate(predictors_test[run_indices_test == run,], targets_test[run_indices_test == run,])
Run number 18
2106/2106 [==============================] - 0s 94us/sample - loss: 201.9923 - mae: 11.6397
Run number 22
3728/3728 [==============================] - 0s 90us/sample - loss: 133.3108 - mae: 9.6191
Run number 28
2175/2175 [==============================] - 0s 90us/sample - loss: 103.5081 - mae: 7.6126

To conclude, the convolutional model achieved best performance on the validation set. It predicts the heart rate on the test cases with an average error of a bit under 10 BPM. The amount of error can be deemed succesful taking into account the different things that can affect running heart rate not included in the predictors. Also, as the heart rate is measured with an optical heart rate sensor from the wrist in differing weather conditions, the measurements might have some inherent error in them. Furthermore, the runs took place in the course of two months and thus there could be a significant difference in my aerobic fitness in the first and last runs of the dataset, affecting the heart rate measurements.